Instability of Probability Laws with Respect to 
Small Violations of Algorithmic Randomness 

Vladimir V. V'yugin* 

Institute for Information Transmission Problems 
Bol'shoi Karetnyi per. 19, Moscow GSP-4, 127994, Russia 
e-mail vyugin@iitp.ru 



Abstract 

We study a stability property of probability laws with respect to 
small violations of algorithmic randomness. A sufficient condition of 
stability of a probability law is presented in terms of Schnorr tests of 
algorithmic randomness. Most probability laws, like the strong law 
of large numbers and the law of iterated logarithm, are stable in this 
sense. 

The phenomenon of instability takes place in ergodic theory. An 
instability of the Birkhoff's ergodic theorem and related asymptotic 
laws with respect to small violations of algorithmic randomness is 
proved in this paper. The Shannon-McMillan-Breiman theorem and 
all universal compression schemes are also instable. 

1 Introduction 



In this paper we study a stability property of probability laws with respect 
to small violations of randomness. By a probability law we mean a property 
^{u) of infinite sequences u which holds almost surely. We define a notion of 
stability of a probability law in terms of algorithmic theory of randomness. 
Within the framework of this theory the probability laws are formulated in 
"a pointwise" form. It is well known that main laws of probability theory 

*This paper is an extended version of the conference paper V'yugin |17j . 
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are valid not only almost surely but for each individual Martin-Lof random 
sequence. 

Some standard notions on algorithmic randomness are given in Sec- 
tion lA.ll We use the definition of a random sequence in the complexity 
terms. An infinite binary sequence U1U2 ... is Martin-Lof random with re- 
spect to uniform (or 1/2-Bernoulli) measure if and only if Km{u"') > n—0{l) 
as n — 7- 00, where Km{u"') is a monotone Kolmogorov complexity of a binary 
string (x)"' = Ui . . . cun and the constant 0(1) depends on co but not on n. 

A probability law $(0;) is called stable if an unbounded computable func- 
tion p{n) exists such that $(0;) is true for any infinite sequence cu such that 
Km{uj^) > n — p{n) — 0(1) as n — > 00. 

A stability property under small violations of algorithmic randomness of 
the main limit probability laws was discovered by Schnorr [lOj and Vovk |13]. 
They shown that the law of large numbers for uniform Bernoulli measure 
holds for a binary sequence U1U2 ... if Km{uj^) > n — cr{n) — 0(1), where 
(y{n) is an arbitrary computable function such that cr^n) = o{n) as — )■ 00, 
and the law of iterated logarithm holds if Km{u^) > n — (j{n) — 0(1), where 
cr(n) is an arbitrary computable function such that (j{n) = o(loglogr;,) as 
n — )• 00. H 

We present in Theorem [1] a sufficient condition of stability in terms of 
Schnorr tests of randomness and prove that most probability laws, like the 
strong law of large numbers and the law of iterated logarithm, are stable 
under small violations of algorithmic randomness. 

In Section [3] we show that the situation is different in ergodic theory. We 
note that there is some analogy with the lack of universal convergence rate 
estimates in ergodic theory. A lack of universal convergence bounds is typical 
for asymptotic results of ergodic theory like Birkhoff's ergodic theorem - 
Krengel Shannon-McMillan-Breiman theorem and universal compressing 
schemes - Ryabko [9j. 

In this paper we present new impossibility results based on analysis of 
ergodic theory in algorithmic randomness framework - instability under small 
violations of algorithmic randomness. 

Some notions of ergodic theory are given in Section IA.3I Let T be a 
measure preserving transformation and / be a real-valued function defined 

^The same property holds also if we replace monotonia complexity Km{Lo") on the 
prefix complexity KP{uj"). A difference is that an inequality Km{uj'^) < n + 0{1) holds 
for monotonia complexity but this is not true for prefix complexity. 
All logarithms are on the base 2. 
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on the set of binary sequences. 

We suppose that the transformation T and the function / are computable 
(see definitions in Section \A.l\i . 

Using Bishop's [1] analysis, V'yugin [15] presented an algorithmic version 
of the ergodic theorem - for any infinite binary sequence u the following 
implication is valid : 

1 n— 1 

Km{u-) >n- 0(1) =^ \im - fi^u^) = /(c^) (1) 

^ 1=0 

for some /(w) (= E{f) for ergodic T). 

Later this result was extended for non-computable / and generalized for 
more general metric spaces. For further development see Nandakumar [S], 
Galatolo et al. [S], and P. Gacs et al. 0. 

We show in Section [3] an instability of this asymptotic result, more cor- 
rectly, that a constant 0(1) in ([1]) cannot be replaced on an arbitrary slow 
increasing computable variable a{n). 

2 Stability of probability laws 

Let fl = {0, 1}°° be the set of all infinite binary sequences. In what follows 
l{a) denotes the length of a finite sequence a. The uniform Bernoulli measure 
is defined L{Ta) = 2^'^°\ where Fq, = {w G ^7 : a C a;} for any finite binary 
string a,. 

In Section [A. li the notions of monotonic complexity Km{x) and of Martin- 
Lof and Schnorr random sequences are defined. 

Let $(a;) be an asymptotic probability law, i.e., a property of infinite 
binary sequences which holds almost surely. 

Kolmogorov's algorithmic approach to probability theory offers a new 
paradigm for logic of probability. The requirement "$(a;) holds almost 
surely" is replaced on a more strong requirement: 

"cj is Martin-Lof random =^ $(0;)". 

For any infinite binary string u, Km{u"') < n + 0(1) for all n. The 
converse inequality holds for any Martin-Lof random sequence u. Namely, an 
infinite sequence u = U1U2 ... is Martin-Lof random if and only if Km{uj^) = 
n + 0(1) as n — )• 00 (see Section IAT]!) . 

Therefore, we can formulate an equivalent form of a probabilistic law: 
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''Km{uj'^) >n- 0(1) as n ->cx) ^ 

In this paper we present a more deep analysis. We call a law ^{u) stable 
if there exists a unbounded nondecreasing computable function a{n) such 
that the following property holds: 

"■Km{u'^) >n - a{n) - 0(1) as n oo ^> $(a;)" 

The function a{n) is called degree of stability of the law <I>(a;). 

We present in this paper a sufficient condition for stability of a probability 
law and prove that most probability laws are stable with different degrees of 
stability. We formulate this condition in terms of Schnorr ^10] definition of 
algorithmic random sequence. See Section lA.ll for details of this definition. 
The choice of Schnorr's definition is justified by an observation that for any 
probability law $(0;) from a vast majority of such laws (like the strong the law 
of large numbers or the law of iterated logarithm) a computable sequence 7^, 
n = 1, 2, . . ., of Schnorr tests of randomness exists such that for any infinite 
sequence $(0;) is true if and only if u passes the test Tk for each k. 

For technical reasons we consider total Solovay tests of randomness which 
leads to the same definition of randomness as Schnorr tests of randomness. A 
computable sequence T = {a;„ :ri = l,2,...}of finite strings is called total 

Solovay test of randomness if the series 2 converges with computable 

n=l 

rate of convergence. 

An infinite sequence u passes a test T if x„ ^ w for almost all n (see a 
more detailed consideration in Section lATT]) . 

In the following theorem some sufficient condition of stability of a prob- 
ability law is given in terms of total Solovay tests randomness. 

By computable sequence of total Solovay tests of randomness we mean 

a computable double indexed sequence of finite binary strings Tk = {xk,n '■ 

00 

n = l,2, ...}, /c = l,2, such that the series 2~^^^'''^> converges with a 

71=1 

uniformly by k computable rate of convergence. This means that there exists 

CO 

a computable function m((5, k) such that 2~^^'='»-' < 5 for each k and 

i=m(<5,fe) 

rational 5. 

Theorem 1 For any computable sequence of total Solovay tests of random- 
ness Tk, k = 1,2, . . ., a computable unbounded function p{n) exists such that 
for any infinite sequence u if Km{u)"') > n — p{n) — 0(1) as n 00 then the 
sequence u passes the test Tk- 
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Proof. Let 71 = {xk^n '■ n = 1,2, . . .} for any k = 1,2, . . .. Since 

oo 
m=l 

with a uniform computable rate of convergence, an unbounded nondecreasing 
computable function z/(n) exists such that 

oo 
m=l 

for any k. By generalized Kraft inequality (see Li and Vitanyi [7]), for any 
k, we can define the corresponding prefix-free code uniformly computable by 
k such that 

Km{Xk,m) < l{Xk,m) - l'{l{Xk,m)) + 0(1) 

Assume an k exists such that Xk,m ^ ^ for infinitely many m. For any such 
m, = Xk,m, where n = l{xk,m)- 

Let p{n) be a unbounded nondecreasing computable function such that 
p{n) = o(z/(n)) as n — )■ oo. Let also, cu be an infinite binary sequence such 
that Km{uj^) > n — p{n) — 0(1) for all n. For some k and n = l{xk,m), 

p{n) >n- Km{u'') >n- l{xk,ra) + v{l{xk,ra)) - 0(1) > v{n) - 0(1) 

for infinitely many n. On the other hand, p{n) = o(z/(n)) as n — ?■ oo. This 
contradiction proves the theorem. A 

Using Theorem [T] we can prove the stability property for main probability 
laws. 

At first, we show that the strong law of large numbers corresponds to an 
infinite sequence of total Solovay randomness tests. 

Hoeffding [3] inequality for uniform probability distribution L says that 
for any e > 



1 " 1 

— > UJi 



> e \ <2e 



(2) 



for all n. 

Let ek be a computable sequence of positive rational numbers such that 
Cfc — i- as A; — )■ oo. Define a sequence of sets 



Uk,n = {x : l{x) 



1 " 1 

— > Xi 

1=1 



(3) 



By 



L{U,^n)= E 2-'W<2e-2-^^ (4) 



for all n and k. 

Let UnUk^n = {a;fc,m : m = 1, 2, . . .}. For any fc, the sequence 

{xk,m : m = 1,2, . . .} 
is a total Solovay test of randomness, since 

OO OO CXD 

E 2"'^"'""^ = E LiUk,n) < E 2e-'"^' < OO 

m=l n=l n=l 

with a computable rate of convergence. 

By definition ([3]) the strong law of large numbers 

1 " 1 
lim — = - 

n-i-oo n ~^ 2 

holds for an infinite sequence u = uiU2 ... if and only if for each k it passes 
the test {xk,m : m = 1, 2, . . .}. 

Then by Theorem [1] an unbounded nondecreasing computable function 
p{n) exists such that if Km{uj^) > n — p{n) — 0(1) as n — )■ oo then for any 
k the sequence u passes the test {xk,m '■ m = 1,2,...}. This means that for 
any k, cu" ^ Uk,n for almost all n, i.e., the strong law of large numbers (E]) 
holds for this u. 

We can find the specific form of this function p{n). Consider any un- 
bounded nondecreasing computable function p{n) such that p{n) = o{n) as 
n — )■ OO. Let also, be a computable sequence of positive rational num- 
bers such that efc — )■ as — )■ OO. It is easy to see that an unbounded 
nondecreasing computable function a{n) exists such that a{n) = o{n) and 
p{n) = o{a{n)) as n — )■ oo. 

We write LZ{Uk,n) < 2-"(")cfc,„, where Ck,n = 2e-2"4+"("). For any k, 

OO OO OO 

J2 2-^M+-(^M) = ^ 2"(")L(f/fc,„) < E Ck,n < OO. 

n=l n=l n=l 

since a{n) = o{n) as n — )■ oo. By the proof of Theorem [1] if Km{u^) > 
n — p{n) then for each k, u;" ^ Un^k for almost all n, i.e., the strong law of 
large numbers holds for this u. 
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An analogous construction can be developed for the law of iterated loga- 
rithm: 



lim sup 



i=l 



A/inlnlnn 



1. 



(5) 



We prove here only the inequality < in ([5]). This inequality violates if and 
only if a rational number 5 > 1 exists such that 



n 



Sr, — — > S\ -n In Inn 



for infinitely many n, where Sn = J2 ^^i- 



i=l 



For any rational number 6 such that 6 > 1 and for rrin = \5^~\ , let [fl 



Us^n — 

= {cj G r2 : 3k{mn < k < m„+i&S'fc — k/2 > 5\J (l/2)m„ In In m„}. (6) 
Using the inequality 

L{ max Sk> a] < 2L{Sm > a}, 

l<k<m 

we obtain 

L{Us,„) < 



< L{{oj e n : 3k{k < rUn+itSk - k/2 > (5A/(l/2)m„lnlnm„)}) < 



< 2L{{uj G ^ : S'm„+i -mn+i/2 > (5y^(l/2)m„ lnlnm„}) < 

1 

n 



< ce-^i'^''^'™" ^ — , (7) 



where c > 0. We have used in ([7]) the Hoeffding inequality (jl]). 

We can effectively construct a prefix-free set Us^n of finite sequences such 
that for each G f/^n an m exists such that G Usn- 



''For any real number r, [r] denotes the least positive integer number m such that 
m > r. 
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A sequence UnUs,n = {xs,k : A; = 1, 2, . . .} is a total Solovay test of 
randomness, since the series 

n n n 

converges (with a computable rate of convergence) for any 5 > 1. 

By definition the law of iterated logarithm ^ holds for uj = UJ1UJ2 ... if 
and only if it passes the test {xs^k : = 1, 2, . . .} for each 6 > 1. 

By Theorem [1] an unbounded nondecreasing computable function p(m) 
exists such that the inequality < in (|5]) holds for any cj satisfying Km{u"^) > 
m — pirn) — 0(1) as m — 7- 00. 

We can also find a specific form of pirn) for the law of iterated logarithm. 
Let p(m) be a unbounded nondecreasing computable function p{m) such that 
p(m) = o(loglogm) as m — )■ cxD. An unbounded nondecreasing computable 
function a{rn) exists such that a(m) = o(loglogm) and pirn) = o(a(m)) as 
m — )■ 00. 

Since Q;(m„) = o(lnlnm„) = o(lnn), the series 

-<51nlnm„+i+a(m„) ^ o(\liin) 



converges for any 6 > 1. By the proof of Theorem [H any sequence u passes 
the test {xg^n : n = 1, 2, . . .} if Kmiu'^) > m — p{m) — 0(1) as m — )■ cxd. 



3 Non-stable probability laws 

The phenomenon of instability takes place in ergodic theory. Some basic 
notions of ergodic theory are given in Section IA.3I In this section we present 
an instability property of the ergodic theorem. 

In Theorems [2] and [3] the uniform measure L and measure preserving 
transformations of f2 are considered. 

Theorem 2 Let cr(n) he a nondecreasing unbounded computable function. 
Then there exist a computable ergodic measure preserving transformation T 
and a sequence u eVL such that the inequality Km{uj^) > n — (j{n) holds for 
all n and the limit 

-| n—l 

jto-E/p-V) (8) 

i=0 
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does not exist for some computable indicator function f (with the range 

{0,1};. 



The construction of the transformation T is given in Section IH the proof of 
Theorem [2] is given in Section O 

In the following theorem an uniform with respect to cr(n) result is pre- 
sented. But in this case, we loss the ergodic property of the transformation 
T. 

Theorem 3 A measure preserving transformation T can be constructed such 
that for any nondecreasing unbounded computable function CT{n) a sequence 
u E fl exists such that Km{u^) > n — a{n) holds for all n and the limit (QP 
does not exist for some computable indicator function f. 

On the proof of this theorem see Section [61 

Does an ergodic transformation T exists satisfying Theorem [3] is an open 
question. 

Note that an infinite sequence u is Martion-Lof random with respect to 
a computable measure P if and only if 

Kmiu"") = -logP(cj") + 0(1) 

as n — )■ oo. 

Recent result of Hochman [3j implies an algorithmic version of the 
Shannon-McMillan-Breiman theorem for Martin-Lof random sequences: for 
any computable stationary ergodic measure P with entropy H, Km{u^) > 
— logP(u;") — 0(1) as n — )■ oo implies 

Kmiu'') -logP(a;") 
lim ^ — - = lim — - = H (9) 

n— >-oo -yi n— ^oo -yi ^ 

The construction given in Section H] shows also an instability property of 
the relation iQ (this was first shown in |16j). 

Theorem 4 Let (j[n) as in Theorem\^ e be a sufficiently small positive real 
number. A computable stationary ergodic measure P with entropy < H < e 
and an infinite binary sequence u exist such that 

Km{uj'') > - log P{uj") - a{n) (10) 
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for all n, and 



limsup ^ — ->-;, (H 

n^oo n 4 



liminf^^^<e. (12) 

The proof of this theorem is given in Section [51 

By a prefix-free code we mean a computable sequence of one-to-one func- 
tions {0n} from {0, 1}" to a prefix-free set of finite sequences. In this case a 
decoding method also exists such that 0„(0ri(Q;)) = a for each a of length 
n. 

A code {(pri) is called universal coding scheme with respect to a class of 
stationary ergodic sources if for any computable stationary ergodic measure 
P (with entropy H) 

lim '<^"<""» = a.s.. (13) 

where l{x) is length of a finite sequence 

Lempel-Ziv coding scheme is an example of such universal coding scheme. 
We have also an instability property for any universal coding schemes. 

Theorem 5 Let (j{n) and e be as in Theorem\^ A computable stationary 
ergodic measure P with entropy < H < e exists such that for each universal 
code {0„} an infinite binary sequence u exists such that 



for all n, and 



Kmiu") > -logP(w") - ain) 



limsup^— ^ — — > -, (14) 

n-i^oo n 4 

lin.inf"^"'-"'><e. (15) 

n— >oo fi 



The proof of this theorem is given in Section |5l 
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4 Construction 



Basic notions of algorithmic randomness theory and ergodic theory are 
given in Sections lA.ll and IA.3I correspondingly. 

We use a cutting and stacking method from Shields [H] and [12] to define 
an ergodic transformation T of the unit interval [0,1). See Section [A. 31 for 
all needed definitions and details of this method. 

Let r > be a sufficiently small rational number. Define a partition 



of the interval [0, 1) (the number r will be specified later). We consider the 
uniform measure A on [0, 1). 

Let cr(n) be a computable unbounded nondecreasing function. A com- 
putable sequence of positive integer numbers exists such that < /i_2 < 
h-i < ho < hi < . . . and 



for all i = 0, 1, . . .. 

The gadgets A^, n^, where s = 0, 1, . . ., will be defined by mathematical 
induction on steps. The gadget Aq is defined by cutting of the interval 
[0.5 — r, 0.5 + r] on 2/io equal parts and by stacking them. Let IIo be a 
gadget defined by cutting of the intervals [0, 0.5 — r) and (0.5 + r, 1] in 2ho 
equal subintervals and stacking them. The purpose of this definition is to 
construct initial gadgets of height 2ho with supports satisfying A(Ao) = 2r 
and A(no) = 1 - 2r. 

The sequence of gadgets {A^}, s = 0, 1, . . ., will define an approximation 
of the uniform Bernoulli measure concentrated on the names ot their trajec- 
tories. The sequence of gadgets {n^}, s = 0, 1, . . ., will define a measure with 
sufficiently small entropy. The gadget Hs-i will be extended at each step of 
the construction by a half part of the gadget A^^i. After that, the indepen- 
dent cutting and stacking process will be applied to this extended gadget. 
This process eventually defines infinite trajectories starting from points of 
the interval [0, 1). The sequence of gadgets {lis}, s = 0, 1, . . ., will be com- 
plete and will define a transformation T of the interval [0, 1) and a measure 
P on these trajectories (defined by f|T8|) . Lemmas [2] and |3] from Section [A. 31 
ensure the transformation T and measure P to be ergodic. 



TTo = [0, 0.5) U (0.5 + r, 1), tti = [0.5, 0.5 + r] 



(16) 



c^ihi^i) - cr(/ii_2) > i - logr + 11 



(17) 
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Construction. Let at step s — 1 (s > 0) gadgets A^-i and n^-i were 
defined. Cut of tlie gadget A^^i into two copies A' and A" of equal widtfi 
(i.e. we cut of each column into two subcolumns of equal width) and join 
n^-i U A" in one gadget. Find a number Rg and do i?s-fold independent 
cutting and stacking of the gadget n^-i U A" and also of the gadget A' to 
obtain new gadgets and A^ of height > 2hs such that the gadget n5_iUA 
is (1 — l/s)-well-distributed in the gadget n^. The needed number Rg exists 
by Lemma [3] (Section \A.3\i . 



Properties of the construction. Define a transformation T = T{Ils} on 
the interval [0, 1). 

Since the sequence of the gadgets {Us} is complete (i.e. X(Jls) 1 and 
w{Ils) — as s — > oo), T is defined for A-almost all points of the unit 
interval. 

The transformation T is ergodic by Lemma[2] (Section |A|) . where = 11^, 
since the sequence of gadgets Lis is complete. Besides, the gadget LIs-i U A", 
and the gadget LI^.i are (1 — l/s)-well-distributed in LI^ for any s. By 
construction A(Aj) = 2~*+V and \(JJi) = 1 — 2~*+-'^r for all z = 0, 1, . . .. 



5 Proof of Theorems [4] and [5 



Recall some notions of symbolic dynamics. Let tt = (7ri,7r2) be a par- 
tition of the unit interval: tti and 7T2 are disjoint and tti U 7r2 = [0, 1). A 
transformation T of the interval [0, 1) defines a measure P on the set of all 
finite and infinite binary sequences: 

P(L J = X{a G [0, 1) : r« G TT,,, i = 1, 2, . . . , n}, (18) 

where A is the uniform measure on [0, 1) and binary se- 

quence. We use notation P{x) = PiT^). 

The measure P can be extended on all Borel subsets of Q by the Kol- 
mogorov's extension theorem. The measure P defined by (IT5]) is stationary 
and ergodic with respect to the left shift if and only if T is a measure pre- 
serving ergodic transformation. 

In the construction of Section H] we have defined a specific partition 
(7ro,7ri) of [0,1) and the the measure preserving ergodic transformation T 
of [0, 1) using a sequence of gadgets A^, LI^, where s = 0, 1, . . .. The union 
of these two gadgets is denoted 



12 



At a step s, we consider an approximation T, = T{$s} of the transfor- 
mation T and the corresponding approximation P"^ of the measure P defined 
by f|T8|) . The transformation Tg determines finite trajectories starting in the 
points of internal intervals of this gadget and finishing in the top intervals. 
Any such trajectory has a name which is a word in the alphabet {0, 1}. By 
definition for any word a (for any set of words D) the number P^{a) {P^[D) 
accordingly) is equal to the sum of lengths of all intervals of the gadget 
from which trajectories with names extending a (extending a word from D) 
start. 

We use the construction of Section H] to suggest conditions under which 
there exists a point in the interval [0, 1) having an infinite trajectory with a 
name a satisfying (ITU]) . (ITT]) and (IT^ . To implement ( TTT]) . we periodically 
extend initial fragments of a by names of trajectories of gadgets As-i (for a 
suitable s). By the incompressibility property most of them have Kolmogorov 
complexity close to its maximal value. 

We use condition f|T7j) and Proposition [3] from Section IA.2I to bound the 
deficiency of randomness of each initial fragment of length n by the value 
(T{n). 

To implement condition (fT2|) . we extend the initial fragments of a in the 
long runs of the construction only in account of names of the trajectories of 
gadgets {lis}, s = 0, 1, . . .. For any s, a portion < r of the support of such a 
gadget belongs to the element tti of the partition. By the ergodic theorem the 
most part of (sufficiently long) trajectories of this gadget visit tti according 
to this frequency. The names of these trajectories have frequency of ones 
bounded by the number 2r from above. This ensures the bound (|T2l) for a 
sufficiently small r. 

This construction is algorithmic effective, so the measure P is computable. 

Let e > be given. Now we prove that entropy H of the measure P is 
less than e if r is sufficiently small. Since A(7ri) = r and the transformation 
T preserves the measure A, by the ergodic theorem almost all points of the 
interval [0, 1) generate trajectories with the frequency r of visiting the element 
TTi of the partition. Thus for any 5 > for all sufficiently large n, the 
measure P of all sequences x of length n with portion of ones < 2r is > 1 — 5. 
Since 2rn < |, we obtain a standard upper bound for Kolmogorov complexity 

For any w 6 [0, 1), this frequency is equal to lim {l/l) X]i=i Xi{T^^) = '^('"'i), where 
Xi(r) = 1 if r e TTi, and Xi(^) — 0> otherwise. 
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of any such x 



K{x) ^ 1 



n 




+ 



2 logn 



n 



< — Srlogr 



(19) 



for all sufficiently large n. By this inequality and by the approximate equality 
dl]), which holds almost surely, we obtain an upper bound H < —Srlogr < e 
for entropy H of the measure P for all sufficiently small r. 

Let us prove that an infinite sequence a exists such that the conclusion 
of Theorem H] holds. We will define a by induction on steps s as the union 
of an increasing sequence of initial fragments 



For all sufficiently large k, the Kolmogorov complexity of the initial fragment 
a{k) will be small if k is odd, and complexity of a{k) will be large, otherwise. 

Define a(0) to be equal to Ilo-name of some trajectory of length > Hq such 
that dp{a{0)) < 2. Such a trajectory exists by Proposition O (Section lA.2p . 
Define s(-l) = s(0) = 0. 

Induction hypotheses. Let a sequence a(0) C . . . C a{k — 1) is already 
defined for some k > 0. 

Suppose that the following properties hold: 

• (i) at some step s{k — l) of the construction the word a{k — l) is equal to 
an Ilg^k-i) - name of a trajectory starting at some point of the support 
of this gadget; 

• (n) this name has length l{a{k — 1)) > hs(^k-i)', 



• (ni) dp{a{k — 1)) < a-{hs(k-2)) — 4 if is odd, and dp{a{k — 1)) < 
a{hsik-2)) and P'^''-^\a{k - 1)) > (l/8)P(a(A; - 1)) if k is even. 



Let us consider any odd k. Define a = a{k — 1). Pick out a set of 
all intervals of the gadget IIs-i such that for any trajectory starting in this 
interval the following properties hold: 

• Ils-i-name of this trajectory extends a; 

• the frequency of visiting the element tti of the partition is < 2r. 



a(0) C . . . C a{k) C . . . 



(20) 
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For any such H^.i-name 7, 

^(7)^(7) < -3r log r < e, (21) 

where r is sufficiently small. The proof of fl21l) is similar to the proof of (I19p . 
By the ergodic theorem, for all sufficiently large s, total length of all intervals 
from this set is > (l/2)P(a). 

Divide all intervals stacking a column of the gadget into two equal 
parts: upper half and lower half. Consider only intervals from the lower 
part. Any trajectory starting from a point of an interval from the lower part 
has length > hg. Fix some sufficiently large s as above and define s{k) = s. 

Let Us{a) be a set of all intervals from the lower part of the gadget 
generating trajectories with Hs-names extending a and satisfying the inequal- 
ity ( I2T]) . By definition the total length of all intervals from Us{a) is more 
than (l/4)P(a). Let Da be a set of all n^-names of these trajectories. 

We can also say that the inequality P^{Da) = P^{a) > (l/4)P(a) holds, 
where we denote D = Ux^d^x for any finite set of strings D. 

It is easy to prove that a set Ca C Da exists such that P{Ca) > 
{l/8)P{Da) and P'{b) > (1/8)P(6) for each b e Ca- 

By Proposition |3] an 6 G exists such that dpilP) < dp{a) + 4 for 
^(a) <j< Kb). Define a{k) = b. 

By induction hypotheses (ii) and (iii) the inequalities dp{a) < o'{hs[k-2)) — 
4 and l{a) > hs{k-i) > hs{k-2) hold. Then dp{b') < a{hs{k-2)) < cr{l{a)) < 
a(j) for all l{a) <]< l{b). 

Notice, that l{b) > hs(k)i since any trajectory with a name b starts from 
an interval of the lower half of the gadget n^, and the height of this gadget 
is > 2hs. Hence, the induction hypotheses (i)-(iii) are valid for the next step 
of induction. 

The condition f|T2|) is true, since condition fl2Tl) holds for infinite number 
of initial fragments a{k) of the sequence a. 

Let k be even. Put b = a{k — 1). Let s = s{k — 1) + 1. Define s{k) = s. 

Consider an arbitrary column of the gadget Ag^i. Divide all its intervals 
into two equal parts: upper half and lower half. Any trajectory starting from 
an interval of the lower part has length > M/2, where M is the height of the 
gadget As-i. By the construction M > 2hs-i. 

Total length of all such intervals is equal to |A(A5_i). 

Let us consider the names x^/'^ of initial fragments of length M/2 of all 
these trajectories. By incompressibility property of Kolmogorov complexity 
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(IHT]) and by choice of M the uniform Bernoulh measure L of all sequences of 
length M/2 satisfying 

is less than 2^^^!^"-'^ < 1/4. The names of initial fragments (of length M/2) 
of the rest part of trajectories starting from intervals of the lower part of the 
gadget A^-i satisfy 

K{x^l^) 2 

-^'1^2 ^^^^ 
By definition for any step s of the construction, the equality 

P'"^{x) = 2-'(^')A(A,_i) 

holds for the name x of any trajectory of the gadget A^-i. We conclude 
from this equality that the uniform measure of all intervals stacked in the 
lower half of the gadget A^-i and generating trajectories with names (more 
correctly, with initial fragments x^^'^ of such names) satisfying (122|) is at 
least j\{As-i). 
By (HZD 

_ X{A") _ A(A,_i) _ 2-^+V 
^ " A(fl,_i) " 2A(fl,_i) " 1 - 2-+2 > 

> 2^^^^r > 2^('^('^''-i)~'^('^''-2)+^^) (23) 

Let us consider i?s-fold independent cutting and stacking of the gadget Hs.iU 
A" in more details. At first, we cut of this gadget on Rg copies. When we 
stack the next copy on already defined part of the gadget the portion of all 
trajectories of any column from the previously constructed part, which go to 
a subcolumn from the gadget A", is equal to 

^(^") - (24) 



A(n,_i) + A(A") 1 + 7 

This is true, since by definition any column is covered by a set of subcolumns 
with the same distribution as the gadget IIs-i U A" has. 

Total length of all intervals of the gadget IIs-i such that trajectories with 
names extending b start from these intervals is equal to P'^~^{b). 
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Consider the lower half of all subintervals obtained by cutting and stack- 
ing of the gadget H^.i in which trajectories with H^.i-names extending b 
start. The length of any such trajectory is at least hg. The measure of all 
remaining subintervals decreases twice. After that, we consider a subset of 
these subintervals, such that trajectories starting from subintervals of this 
subset go into subcolumns of the gadget A". The measure of the set of re- 
maining subintervals is multiplied by a factor 7/(1 + 7). Further, consider 
the subintervals from the remaining part generating trajectories whose names 
have in A" fragments satisfying ( 122|) . The measure of the remaining part can 
be at least 1/4 of the previously considered part. This follows from the pre- 
vious estimate of the portion of subintervals generating trajectories in the 
gadget A" of length > M/2 satisfying (12^ . Let Df, be a set of all rig-names 
of all trajectories starting from subintervals remaining after these selection 
operations. Then 

nn,) > ^^P'-\b). (25) 

The name of any such trajectory has an initial fragment of type bx'x^^'^, 
where x'x*^/^ is the name of a fragment of this trajectory corresponding to 
its path in the gadget As_i. The word x*^/^ has length M/2 and satisfies 
fl22|) . The word x' is the name of a fragment of the trajectory which goes 
from an interval of the lower part to an interval generating trajectory with 
the name x^/^. We have l{bx'x^^/^) < 2M = Al{x^'^/^). By ([22D and we 
obtain for these initial fragments of sufficiently large length: 

Kibx'x^^l^) K{x^^/^)-2\ogl{bx') 1_ 1 
l{bx'x^^/^) - 4/(a;*^/2) " 4 " 7^' ^^^^ 

We have P'^^^{b) > (1/8)P(6) by induction hypothesis (iii). After that, 
taking into account that 7 < 1, we deduce from (1251) 

P{Db) > P'~\D,) > ^P{b). 

By Proposition |3] an c G -Dfc exists such that 

dp{ci) <dp{b) + I -log ^< 

< dp{b) + {a{hs-i) - a{K-2) - 12) + 8 < 
< a{hs-i) - 4 = a{hs(k~i)) - 4 
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for all l{b) < j < /(c). Here we have dp{b) < (y{hs(k^2)) ^ <^{hs-2) by in- 
duction hypothesis (iii). We also used inequality ( 12^ . Besides, by induction 
hypothesis (ii) we have l{h) > hg^i. Therefore, 

dp(c') < (t(/i,_i) < am) < ^(J) 

for l{b) < j < l{c) . Define a{k) = c. 

It is easy to see that all induction hypotheses (i)-(iii) are valid for a{k). 

An infinite sequence a is defined by a sequence of initial fragments (|20|) . 
We have proved that dp{a^) < a{j) for all j > l{a{l)). 

By the construction there are infinitely many initial fragments of the 
sequence a satisfying (!26|) . The sequence /is, where s = 0, 1, . . ., is monotone 
increased. So, the condition (fTTj) hold. A 

We complete this section by the proof of Theorem [51 

Since any universal coding scheme is asymptotically optimal, i.e., for 
almost all sequences, it has compressing ratio (IT^ equals to Kolmogorov 
complexity compressing ratio ([9]), we obtain an instability property for any 
universal coding scheme. 

In more details, for any n a decoding algorithm ipn of the code is 
defined by a program of length \ogn + 0(1). Then we have an upper bound 
for Kolmogorov complexity of any initial fragment of the sequence a defined 
in the proof of Theorem [S] 

i^(a") </(</>«(«")) + 0(logn). 

Inequality f[T^ follows from the inequality flTT]) of Theorem |H The proof of 
inequality (fT5!) is similar to the proof of inequality (fT2l) of Theorem HI In 
that proof we replace condition (pT!) on /(0„(u;"))/?2 < e. We also take into 
account the property (fT3l) of asymptotic optimality of the code {0ri}- 



6 Proof of Theorem [2 



To prove Theorem [2] we interpret T as a transformation of infinite bi- 
nary sequences preserving the measure L on Vt. To do this we will use a 
natural correspondence between points of [0, 1) and infinite binary sequences 
representing dyadic representations of these points (see Section H]). This 
correspondence is one-to-one besides a set of measure zero. 
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We consider also a natural correspondence between finite binary se- 
quences and subintervals of [0, 1) with dyadically rational endpoints. A finite 
sequence a of length n corresponds to an interval [a, + 2"") with dyadically 
rational endpoints, where a = Si<i<nCKi2~*. Also, any sequence of strictly 
nested intervals with dyadically rational endpoints corresponds to an infinite 
binary sequence. 

When stacking a subinterval J of [0, 1) on a subinterval / we simply add to 
each point a G / a rational number g, i.e., to define Ta = a + g & J . We can 
interpreted this transformation T as a computable operation transforming 
any binary representation of a number a to a binary representation of the 
number a + q. This operation transforms any binary representation 
of the number a from below up to 2~" to a binary representation of a + g from 
below up to 2-("-^). We use a sufficiently accurate binary representation of 
the rational number q. 

In what follows we mean by T the corresponding computable operation 
on VL. 

We use the construction of Section H] to show that an infinite sequence 
w G exists such that Km{u'^) > n — (j{n) for all n and the limit does 
not exist for the name x(a;)x(Tcj)x(T^(X') ... of its trajectory, where x{^) = i 
if w G TTj, i = 0, 1. 

We will prove that an w G f2 exists such that Km{ijj^) > n — <y{n) for all 
n and the limit ^ does not exist for the name x{!^)xiTu))x{T'^^) ... of its 
trajectory, where x{'^) = if w G VTj, i = 0, 1. More precise, we prove that 

n— 1 

limsup-^x(T*c^)>l/16, (27) 

1 n— 1 

ll^^f ^ E ^(^^^) ^ 2r, (28) 

where r is sufficiently small and an indicator function x is defined below. 
We will define by induction on steps s a sequence uj satisfying the conclu- 
sion of Theorem [2] as the union of an increasing sequence of initial fragments 

w(0) C . . . C io{k) C . . . (29) 

Using Proposition [3] (Section [Al2l) . where P is the uniform measure A, define 
a;(0) such that rf(u;(0)-') < 2 for all j < /(a;(0)). Define s(-l) = s(0) = 0. 



^We suppose that the rational number r from the definition of the partition ttq, tti has 
a dyadic rational form. 
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Induction hypotheses. Suppose that A; > and a sequence c<;(0) C . . . C 
u{k — 1) is already defined. Also, the interval with dyadically rational end- 
points corresponding to u{k — 1) is a subset of an interval from the support 
of the gadget Ils(^k-i)- We suppose that 

• (i) l{u{k - 1)) > 

• (ii) d{uj{k — 1)) < cr(/is(jfc_2)) —3 if A; is odd and d{uj{k — 1)) < cr{hs[k~2)) 
if k is even, where = n — Km{uj^) is the deficiency of randomness. 

Consider an odd k. Denote a = uj{k — 1). 

Let us consider a sufficiently large s and a set of all finite strings extending 
a such that the corresponding intervals with dyadically rational endpoints 
are subsets of intervals from columns of the gadget ns_i and generate IIs-i- 
trajectories with frequency of visiting the element ttq of the partition less 
than 2r. By the ergodic theorem the total measure of these subintervals 
tends to 2~'(") as s — )■ oo. 

We consider a sufficiently large s such that the total measure of the set 
of all intervals with dyadically rational endpoints locating in the lower half 
of the gadget II,, is > (1/4)2""^''^ Let Ca be a set of strings corresponding to 
these subintervals. 

Fix some such s and define s{k) = s. 

By Proposition [3] (Section IA.2p . where P = L, an b E Ca exists such 
that d{¥) < d{a) + 3 for each l{a) < j < lib). Define u{k) = b. By 
induction hypotheses (i)-(ii) inequalities d{a) < o'{hs[k~2)) ~ 3 and Z(a) > 
hs(k-i) > hs{k-2) hold. Then d{b>) < (T{hs(k~2)) < cr(Ua)) < o-(j) for all 
^(ct) < j < K^)- Also, we can take Z(6) > hs[k)- Therefore, the induction 
hypotheses and condition (1251) are valid for the next step of induction. 

Let k be even. Put b = u{k — 1) and s = s{k — 1) + 1. Define s{k) = s. 

Let us consider an arbitrary column of the gadget A^-i. Divide all its 
intervals into two equal parts: upper half and lower half. Any interval of the 
lower part generates a trajectory of length > M/2, where M > 2/is_i is the 
height of the gadget A^-i. The uniform measure of all such intervals is equal 
to |A(A,_i). 

By Hoeffding inequality ([2]) the measure A of all points of support of the 
gadget As_i whose trajectories have length > M/2 and frequency of ones 
< 1/4 is less than 

2-^^^A(A,_i) < ^A(A,_i) 
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(assume that M is sufficiently large). Then all such points from intervals of 
the lower part of the gadget As_i whose trajectories a have length > M/2 
and frequency of ones more than 3/4 have total measure at least |A(As_i). 
By (HID 

_ _ A(A,_i) _ 

^ " A(fl,_i) " 2A(fl,_i) " 1 - 2-+2 > 

Let $5 is the gadget generated by cutting and stacking of H^^i U A". 

Consider a set of all binary strings correspondent to intervals from the 
lower part of $s such that trajectories starting from these intervals and ex- 
tending the sequence b pass through an upper subcolumn of the gadget A" 
and have frequencies of ones at least 1/4. Notice that any copy of the gadget 
A" has the same frequency characteristics of trajectories. 

Let Db be a set of all binary strings correspondent to these intervals. By 
definition trajectory of any such interval has length at most 2M and some 
initial fragment of its name has at least M/ 8 ones. Hence, frequency of ones 
in the name of any such trajectory is at least ^. 

Since 7 < 1, the total measure of all such intervals is at least ^2~'-^''\ By 
Proposition [3l where P = A, an c G -Dfc exists such that 

d{c^)<d{b) + l-\og^< 
< d{b) + ((t(/i,_i) - a{K_2) - 12) + 6 < 

for all j such that l{b) < j < l{c). Here we have d{b) < cr(/is(fc-2)) < c"(^s-2) 
by induction hypothesis (ii). We also have used the inequality (15U]) . Besides, 
by the induction hypothesis (i) we have l{b) > hg-i- Therefore, 



d{c^) < a(/i,_i) < am) < ^(J) 



for all j such that l{b) < j < l{c). Define u{k) = c. It is easy to see that 
induction hypotheses (i)-(ii) are vahd for u{k). 

An infinite sequence u is defined by a sequence of its initial fragments 
fl29|) . We have proved that d{u^) < a{j) for all j. 

By the construction there are infinitely many initial fragments of trajec- 
tory of the sequence u with frequency of ones > 1/16 in their names. Hence, 
the condition fl27|) holds. A 
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The proof of Theorem [3] is more comphcated. Consider a sequence of pair- 
wise disjoint subsets T^.-, where Xj = T^^O, such that L^UiT^-) = 1. For any 
Txi, we apply the construction of Section H] to a computable sequence ai{n) 
of all partial recursive functions and define a computable ergodic measure 
preserving transformation Tj on F^. for each i. The needed transformation 
is defined as union of all these transformations Tj. We pass details of this 
construction. 

A Auxiliary notions and assertions 
A.l Algorithmic randomness 

Let = {0, 1}* be a set of all finite binary sequences (strings) and fl = 
{0, 1}°° be a set of all infinite binary sequences. Let /(a) denotes the length 
of a sequence a {l{a) = oo for a G 

For any finite or infinite sequence u = (jJiuj2 . . ., we write cu" = ujiuJ2 ■ ■ ■ oon, 
where n < l{u). Also, we write a C /3 if a = /3" for some n. Two finite 
sequences a and /3 are incomparable if a ^ /3 and P ^ a. A set A C B is 
prefix-free if any two sequences from A are incomparable. 

A complexity of a word x G 6* (with respect to a word y G 0*) is equal 
to the length of the shortest binary codeword p (i.e. p G {0, 1}*) by which 
given y the word x can be reconstructed: 

K-4,{x\y) = mm{l{p) : V^(p,y) = x}. 

We suppose that min0 = +oo. 

By this definition the complexity of x depends on a computable (partial 
recursive) function ip - method of decoding. Kolmogorov proved that an 
optimal decoding algorithm ip exists such that 

K4x\y)<K^ix\y) + 0{l) 

holds for any computable decoding function ip' and for all words x and y. 

We fix some optimal decoding function ip. The value K{x\y) = K^{x\y) 
is called (conditional) Kolmogorov complexity of x given y. Unconditional 
complexity of x is defined K{x) = K{x\A). 

We will use some properties of Kolmogorov complexity from Li and Vi- 
tanyi [7j. Incompressibility property asserts that for any positive integer 
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numbers n and m a portion of all sequences x of length n such that 

K{x) < n — m, (31) 

is less than 2~™. Indeed, the number of all x satisfying this inequality does 
not exceed the number of all binary programs generating them. Since the 
length of any such program is less than n~m the number of these programs 
is less than 2""™. 

Let X and b be finite words. It is easy to construct a function which 
given any program computing bx and the length of b computes the word x. 
Therefore, we obtain an extension property 

K{x) <K{bx) + 2logl{b)+c (32) 

for any x, where c is a positive constant not depending from b and x. 

We use a probability space {Q, V, P), where Q = {0, 1}°° is the set of all 
infinite binary sequences and V is the set of all Borel subsets Q. A probability 
measure P on is defined as follows: define P{Ta) for each finite sequence 
a, where = {uj E Q : a G u}, and extend it for all Borel subsets of Q. 

Let 7^ be a set of all real numbers, Q be a set of all rational numbers. 

In this paper we consider computable measures on Q. A function / : 6 — >^ 
TZ is called computable if there exists an algorithm which given a; G O and 
a rational e > computes a rational approximation of a number f{x) with 
accuracy e. 

An important example of computable measure is the uniform Bernoulli 
measure. The uniform Bernoulli measure on Q is defined 

L(r„) = 2-'(") 

for any finite binary sequence a. 

An open subset U of fl is called effectively open if it can be represented as 
a union of a computable sequence of intervals: U = U^^Fa,, where = f{i) 
is a computable function from i. A sequence f/„, n = 1, 2, ... of effectively 
open sets is called effectively enumerable if it can be represented as f/„ = 
U^iFa^i, where an,i = f{n,i) is a computable function from n and i. 

Martin-Lof test of randomness with respect to a computable measure P 
is an effectively enumerable sequence f/„, n = 1, 2, . . ., of effectively open sets 

^ Recall that we consider in the following logarithms on the base 2. 
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such that P{Un) < for all n. If the real number P{Un) is computable 
then the test n = 1, 2, .... is called Schnorr test of randomness. 

An infinite binary sequence u is called Martin-Lof random with respect 
to P if a; ^ r^Un for any Martin-Lof test of randomness n = 1, 2, . . .. A 
notion of Schnorr random sequence is defined analogously. 

An equivalent definition of randomness can be obtained using Solovay 
tests of randomness. A computable sequence {a;„ : n = 1, 2, . . .} of finite 

strings is called Solovay test of randomness if the series 2 converges. 

n=l 

An infinite sequence uj passes a Solovay test of randomness {a:„ : n — 
1,2,.. .} ii Xn ^ oj for almost all n. 

We have used an equivalence between Martin-Lof and Solovay tests of 
randomness. 

Proposition 1 An infinite sequence u = U1LO2 . . . is Martin-Lof random if 
and only if it passes each Solovay test of randomness. 

Proof. Assume that cu is not Martin-Lof random. Then a Martin-Lof test 
n = 1, 2, . . ., exists such that u G nf/„. Define a Solovay test of randomness 
as follows. Since L{Un) < 2"" for all n, we can effectively compute a prefix- 
free sequence of finite strings n = 1,2,..., such that LlnTx„ = UnUn and 

the series J2 2 converges. Then x„ C a; for infinitely many n. 

n=l 

On the other side, assume that for some Solovay test x„, n = 1, 2, . . ., 

00 

Xn d UJ for infinitely many n. Let J2 2^'^^"^ < 2^ , where X is a positive 

n=l 

integer number. Let Un be a set of all infinite cu such that \{m : a;^^ C a;}| > 
2"+^. It is easy to verify that C/„ is a Martin-Lof test of randomness and 
that oj e nUn. A 

We strengthens the definition of Solovay test of randomness - we consider 

total Solovay tests of randomness. 

00 

A series J2 converges with a computable rate of convergence if a com- 

i=l 

00 

putable function m{5) exists such that | X) rj| < 5 for each positive rational 

i=m{5) 

number 5. 

A computable sequence {xn : n = 1, 2, . . .} of finite strings is called total 
Solovay test of randomness if the series 2 converges with computable 

n=l 

rate of convergence. 
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Proposition 2 An infinite sequence uj = UJ1UJ2 . . . is Schnorr random if and 
only if it passes each total Solovay test of randomness. 

The proof is similar to the proof of Proposition [TJ 

Using some modification of decoding algorithms, we obtain an equivalent 
definition of the notion of algorithmic random sequence in terms of algorith- 
mic complexity (see Li and Vitanyi [7]). 

Let us define a notion of a monotonic computable transformation of binary 
sequences. A computable representation of such an operation is a set C 
{0, 1}* X {0, 1}* such that 

• (i) the set is recursively enumerable; 

• (ii) for any (x, y), (x', y') E -ijj if x (1 x' then y C or C y; 

• (iii) if (x, y) E 'ip then (x, y') G -ip for all y' C y. 

The set ^/^ defines a monotonic with respect to C decoding function 

ip{p) = sup{x : 3p'{p' C pk.{p\ x) G "ip)}. (33) 

Any computable monotonic function ip determines the corresponding mea- 
sure of complexity 

Km^{x) = min{Z(p) : x C 'pi^p)} = min{/(p) : (x,p) G ■?/'}. 

The invariance property also holds for monotonic measures of complexity: an 
optimal computable operation ip exists such that Km^{x) < -R'm^/(x) + 0(1) 
for all computable operations %p' and for all finite binary sequences x. 

The corresponding optimal complexity Km{x) = Km^{x) differs from 
simple Kolmogorov complexity K{x) by a term of order 0(log/(x)) 

K{x) < Km{x) + 0(1) < K{x) + 0(log/(x)) 

An infinite sequence u is Martin-Lof random with respect to a computable 
measure P if and only if 

irm(u;") = - log P(u;") + 0(1) (34) 
^ Here the by supremum we mean an union of all comparable x in one sequence. 
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as n — 7- oo. In particular, an infinite binary sequence oj is Martin-Lof random 
(with respect to uniform measure) if and only if Km{u^) = n + 0(1) as 
n — )■ cxD (see for details Li and Vitanyi |7j). 
A function 

cip(a;") = -logP(w") - irm(w") 

is called universal deficiency of randomness (with respect to a measure P). 
For the uniform measure, d{u") = n — Km{u"'). 

By flM|) . dp{u^) < oo if and only if a sequence u is Martion-Lof random 
with respect to P. 

A transformation T of the set Q is computable if a computable represen- 
tation ip exists such that (i)-(iii) hold and 

T{uj) = sup{y : x C u^{x,y) G il>)} (35) 

for all infinite u ^ Q. 

Recall that Q denotes the set of all rational numbers, and let r denotes 
an arbitrary element of Q. 

A nonempty set is called recursively enumerable if it is a range of some 
computable function defined on the set of positive integer numbers. 

Let / be a function from f2 to 7^ U {— C)0, +oo}. A function / : 6 — )■ Q is 
called lower approximation of / if r < /(w) if and only if r < f{x) for some 
X C u. 

A function / is lower semicomputable is it has a computable lower ap- 
proximation. This definition means that if r < f{uj) this fact will sooner or 
later be learned by some algorithm. 

A function / is called upper semicomputable if the function — / is lower 
semicomputable. |^ A function / : — > 7^ U {— oo, +00} is computable if it 
is lower and upper semicomputable. 

A. 2 Bounded increase of deficiency of randomness 

In the proof of Theorem H] a property of a bounded increase of the deficiency 
of randomness was used. Let P be a measure, P{x) 7^ and a set A consists 
of words y such that x C 

Let A = U{Ty : y e A} for any A C {0,1}*. Define P{A\x) = 
P{A)/P{x). 

®Aii equivalent definition can be defined in terms of computable upper approximation. 
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Proposition 3 Let P be a computable measure, x be a word, P{x) ^ and 
a set A consists of words y such that x C y and P{A) > 0. Then for any 
< yU < 1 a subset A' ^ A exists such that P{A') > fiP{A) and 

dp{yn < dp{x) - log(l - /i) - log P{A\x) 

for all y E A' and l{x) <n< l{y). 

Proof. We will use in the proof a notion of supermartingale [13]. A func- 
tion M is called P-supermartingale if it is defined on {0, 1}* and satisfies 
conditions: 

• M(A) < 1; 

• M{x) > M{xO)P{0\x) + M{xl)P{l\x) for all x, where P{u\x) = 
P{xi')/P{x) for 1/ = 0, 1 (we put here 0/0 = * cxd = 0). 

A supermartingale M is lower semicomputable if the set {(r, a;) : r < M(a;)}, 
where r is a rational number, is a range of some computable function. We 
will consider only nonnegative supermartingales. 

Let us prove that the deficiency of randomness is bounded by a logarithm 
of some lower semicomputable supermartingale. 

Lemma 1 Let P be a computable probability measure. Then there exists a 
lower semicomputable P-supermartingale M such that dp{x) < logM(a;) for 
all X. 

Proof. Let some optimal function ip satisfying ( 133|1 defines the monotone 
complexity Km{x). Define 

Q{a) = LiU{r,:aC^{p)}), (36) 

where L is the uniform Bernoulli measure. It is easy to verify that Q{A) < 1 
and Q{a) > Q{aO) + Q{al) for all words a. Then the function M{a) = 
Q{a) / P{a) is a P-supermartingale. 

Since for any a the shortest p such that a C ?/)(p) is an element of the set 
from (|36|) . we have inequality Q{a) > 2^^"^^"\ and so, dp{a) < log M(a;). 
A 

We have dp{x) < logM(x), where M is lower semicomputable P- 
supermartingal. Define a set 

A, = {yeA: 3j{l{x) < j < l{y)kM{y^) > M(x)/((1 - fi)P{A\x)))}. 
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By definition of supermartingale for any prefix free set B such that x C y 
for all ?/ G inequality 

M{x) > M{y)P{y\x) (37) 

holds. For any y & Ai let y^ be the initial fragment of y of maximal length 
such that > (i_^)p(^|a;) • The set {y^ : y E Ai} is prefix free. Then by 

(!37|) we have 

> = — P{Ai\x). 

- (l-^)P(A|x) ^ " ^ 

From this we obtain P{Ai\x) < (1 — fi)P{A\x). Define 

A' = A — {y E A : z C y ioT some z G Ai}. 

Then P(yl'|x) > iJ,P{A\x). For any y G A' we have 

M(y^) < Mix) — = — 

for all l{x) < j < {y). The proposition follows from the inequality dp{x) < 
logM(x). A 



A. 3 Method of cutting and stacking 

An arbitrary measurable mapping of the a probability space into itself is 
called a transformation. A transformation T preserves a measure P if 
P(T~^{A)) = T{A) for all measurable subsets A of the space. A subset 
A is called invariant with respect to T if T~^A = A. A transformation T is 
called ergodic if each invariant with respect to T subset A has measure or 1. 

Recall the main notions and properties of cutting and stacking method 
(see Shields [HI [I2])- A column is a sequence E = (Li, . . . , Lh) of pairwise 
disjoint subintervals of [0,1) of equal width (and with rational endpoints); 
Li is the base, Lh is the top of the column, E = U^^^Lj is the support of 
the column, w{E) = A(Li) is the width of the column, h is the height of the 
column, X{E) = A(U^^]^Lj) is the measure of the column. 
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Any column defines an algorithmically effective transformation T which 
hnearly transforms Lj to Lj^i for all j = 1, . . . , h — 1. This transformation 
T is not defined outside all intervals of the column and at all points of the 
top Lh interval of this column. Denote T^u = u, T^~^^uj = T{T^u). For 
any 1 < j < h an arbitrary point u G Lj generates a finite trajectory 



Since all points of the interval Lj generate the identical trajectories, we 
refer to this trajectory as to trajectory generated by the interval Lj. 

A partition vr = (tti, . . . , TCk) is compatible with a column E if for each 
j there exists an i such that Lj C vTj. This number i is called the name 
of the interval Lj, and the corresponding sequence of names of all intervals 
of the column is called the name of the column E. For any point u G Lj, 
where 1 < j < h, hj ii^-name of the trajectory u, Tu, . . . , T^~^uj we mean a 
sequence of names of intervals Lj, . . . ,Lh from the column E. The length of 
this sequence is h — j + 1. 

A gadget is a finite collection of disjoint columns. The width of the gadget 
w{T) is the sum of the widths of its columns. A union of gadgets Tj with 
disjoint supports is the gadget T = UTj whose columns are the columns of 
all the Tj. The support of the gadget T is the union T of the supports of all 
its columns. A transformation T(T) is associated with a gadget T if it is the 
union of transformations defined on all columns of T. With any gadget T 
the corresponding set of finite trajectories generated by points of its columns 
is associated. By T-name of a trajectory we mean its £^-name, where E is 
that column of T to which this trajectory corresponds. A gadget T extends 
a column A if the support of T extends the support of A, the transformation 
T(T) extends the transformation T(A) and the partition corresponding to T 
extends the partition corresponding to A. 

The cutting and stacking operations that are common used will now be 
defined. The distribution of a gadget T with columns Ei, . . . , En is a vector 
of probabilities 



A gadget T is a copy of a gadget A if they have the same distribution and 
the corresponding columns have the same partition names. A gadget T 
can be cut into M copies of itself T^, m = 1, . . . , M, according to a given 
probability vector (71,..., 7^) of type f l38|) by cutting each column Ei = 



u,Tu,..., T^-^uj. 




(38) 
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{Lij : 1 < i < h{Ei)) (and its intervals) into disjoint subcolumns = 
(^i,j,m : 1 < J < /i(-Ei)) such that w{Ei^rn) = w{Lij^rn) = lmw{Lij). The 
gadget Tm = {Ei^m : 1 < i < L} is called the copy of the gadget T of width 
■jra- The action of the gadget transformation T is not affected by the copying 
operation. 

Another operation is the stacking gadgets onto gadgets. At first we con- 
sider the stacking of columns onto columns and the stacking of gadgets onto 
columns. 

Let El = {Lij : I < 3 < h{Ei)) and E2 = (Lsj : I < J < KE2)) be 
two columns of equal width whose supports are disjoint. The new column 
El* E2 = {Lj : 1 < j < h{Ei) + h{E2)) is defined as Lj = Lij for all 
1 < J < h{Ei) and Lj = L2^j-h(E^)+i for all h{Ei) <j< h{Ei) + h(E2). Let 
a gadget T and a column E have the same width, and their supports are 
disjoint. A new gadget E *T is defined as follows. Cut E into subcolumns 
Ei according to the distribution of the gadget T such that w{Ei) = 'w{Ui), 
where Ui is the i-th column of the gadget T. Stack C/j on the top of E^ to 
get the new column Ei * Ui. A new gadget consists of the columns {Ei * Ui). 

Let T and A be two gadgets of the same width and with disjoint supports. 
A gadget T * A is defined as follows. Let the columns of T are (Ei). Cut A 
into copies Aj such that w{Ai) = w{Ei) for all i. After that, for each i stack 
the gadget Aj onto column Ei, i.e. we consider a gadget Ei * Aj. The new 
gadget is the union of gadgets Ei*Ai for all i. The number of columns of the 
gadget T * A is the product of the number of columns of T on the number 
of columns of A. 

The M-fold independent cutting and stacking of a single gadget T is 
defined by cutting T into M copies Tj, i — 1, . . . , M, of equal width and 
successively independently cutting and stacking them to obtain T*^^) = 
Ti * . . . * Tm- 

A sequence of gadgets {T^} is complete if 

• lim u'(T^) = 0; 

m—^oo 

• lim A(t^) = 1; 

m— >-oo 

• T^+i extends for all m. 

Any complete sequence of gadgets {T^} determines a transformation T — 
T{Ts} which is defined on interval [0, 1) almost surely. 
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By definition T preserves the measure A. In [TT] the conditions sufficient 
a process T to be ergodic were suggested. Let a gadget T is constructed by 
cutting and stacking from a gadget A. Let ii^ be a column from T and D be 
a column from A. Then E (1 D is defined as the union of sub columns from 
D of width w{E) which were used for construction of E. 

Let < e < 1. A gadget A is (1 — e)-well-distributed in T if 

Y: E \KEnD)-X{E)Xm<e. (39) 

i?eA-BGT 

We will use the following two lemmas. 

Lemma 2 (ITl^. Corollary 1), (IT^ . Theorem A. 1). Let {Tn} be a complete 
sequence of gadgets and for each n the gadget {Tn} is {1 — en) -well-distributed 
in {T„+i}, where — )■ 0. Then {T„} defines the ergodic process. 

Lemma 3 (JT^J, Lemma 2.2). For any e > and any gadget T there is an 
M such that for each m > M the gadget T zs (1 — e) -well- distributed in the 
gadget T*^™) constructed from T by m-fold independent cutting and stacking. 
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